On Retaining Intermediate Probabilistic Models When Building Bayesian Networks

The process of building a Bayesian network may occur in stages, in which intermediate Bayesian networks are built during preliminary processing and then used in the construction of further Bayesian networks. For example, in (Doshi, Greenwald, & Clarke 2001) we describe a way to use Bayesian networks to model and correct errors in noisy datasets. The corrected datasets are then used in (Doshi 2001) to build predictive Bayesian networks. Through this process we built networks that capture probabilistic relationships between 412 fields of data from 169,512 patients admitted to trauma centers in Pennsylvania and registered in the Pennsylvania Trauma Systems Foundation Trauma Registry between 1986 and 1999. In the process mentioned above, intermediate Bayesian networks were used to find the most likely values for fields found to have errors. These most likely values were then used in the cleansed dataset. However, in the subsequent process of building Bayesian networks from this dataset, we questioned whether or not these intermediate networks used in error correction should have been retained. In other words, we wanted to understand the tradeoffs involved in retaining the distributional information summarized in each error-correction network rather than just retaining the most likely value for each corrected field. This question can be generalized to any process of building a Bayesian network in stages. This note describes preliminary work to understand these issues. An important component of this staged network building process is that common variables are represented from one stage to the next. In data cleansing, variables used to query for error distributions are the same variables that are used as evidence variables in the final predictive network. Furthermore, the context variables used to model errors are also represented directly in the final network. Retaining distribution information can be accomplished by employing networks from early stages within the subsequent networks. Common variables limit the potential blow-up in network size.